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Abstract 

We study the joint source channel coding problem of transmitting an analog source over a Gaussian channel in 
two cases - (i) the presence of interference known only to the transmitter and (ii) in the presence of side information 
known only to the receiver. We introduce hybrid digital analog forms of the Costa and Wyner-Ziv coding schemes. 
Our schemes are based on random coding arguments and are different from the nested lattice schemes by Kochman 
and Zamir that use dithered quantization. We also discuss superimposed digital and analog schemes for the above 
problems which show that there are infinitely many schemes for achieving the optimal distortion for these problems. 
This provides an extension of the schemes by Bross et al to the interference/side information case. We then discuss 
applications of the hybrid digital analog schemes for transmitting under a channel signal-to-noise ratio mismatch 
and for broadcasting a Gaussian source with bandwidth compression. 

I. Introduction and Problem Statement 

Consider the classical problem of transmitting K samples of a discrete-time independent identically distributed 
(i.i.d) real Gaussian source v in N uses of an additive white Gaussian noise (AWGN) channel such that the mean- 
squared error distortion is minimized. Let the source be encoded into the sequence x which satisfies a power 
constraint B[xx T ] < NP. Let us first consider the case of K = N and let the output of the AWGN channel y be 
given by 

y = x + w 

where w is a noise vector of i.i.d Gaussian random variables with zero mean and variance a 2 . If the source 
variance is a%, then the optimal mean-squared error distortion that can be achieved is D opt = . This optimal 
performance can be achieved by two very simple schemes. The first one is separate source and channel coding, 
where the source is first quantized and the quantization index is transmitted using an optimal code for the AWGN 
channel. The second scheme is uncoded (analog) transmission with power scaling [3,4], where the source is not 
explicitly quantized. Recently, it was shown by Bross, Lapidoth and Tinguely [6] that there is a family of infinitely 
many schemes that are optimal, which contains the separation based scheme and uncoded (analog) transmission as 
special cases. 
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In this paper, we consider the problem of transmitting K samples of an i.i.d Gaussian source through N uses of 
an AWGN channel. We will refer to the ratio of N/K as the bandwidth efficiency A. We first consider the case 
of A = 1 and the presence of an interference known only to the transmitter and/or side information available only 
at the receiver. We derive hybrid digital analog (HDA) coding schemes for these cases where the source is not 
explicitly quantized and show that they can obtain the optimal distortion. These can be viewed as the equivalent 
of uncoded transmission but in the presence of an interference or side information. Then, we show that there is a 
family of infinitely many schemes that are optimal for this problem which contain pure separation based schemes 
and HDA schemes as special cases. This can be viewed as the extension of Bross, Lapidoth and Tinguely's [6] 
result in the presence of interference/side-information. An interesting aspect of the hybrid digital analog coding 
schemes proposed here is that they do not require binning unlike their separation based counterparts. 

The HDA scheme proposed here for the case of interference known at the transmitter is closely related to the 
scheme considered by Kochman and Zamir in [7], although this was developed independently. The difference is 
that the proposed scheme is based only on random coding arguments and does not use nested lattices like in [7]. 
As a result, the relationship between the auxiliary random variable and the source is made more explicit. We also 
consider several applications of the HDA schemes which are not considered in [7]. We consider the non-asymptotic 
SNR case unlike in [7]. Further, the performance of this scheme in the presence of SNR mismatch is analyzed. 
Finally, the use of a HDA Costa based scheme for broadcasting a Gaussian source to two users with bandwidth 
compression, where A < 1 is discussed. In the case of side-information available only at the receiver, the proposed 
scheme is similar to the scheme in [2] and again uses random coding arguments instead of nested lattices. 

The paper is organized as follows. First in Section [TTJ we discuss the problem of transmitting an i.i.d Gaussian 
source in the presence of a Gaussian interference known only to the transmitter. We introduce a hybrid digital 
analog (HDA) Costa coding scheme where the source is not explicitly quantized and show that this is optimal. We 
then discuss a generalized HDA Costa coding scheme and show that there infinitely many schemes that are optimal. 
In Section |in| we discuss similar schemes and results for the case of having side information available only at the 
receiver (Wyner-Ziv problem) and in Section [IVJ briefly consider the situation having interference known only to the 
transmitter and side-information available only at the receiver. In [11], Merhav and Shamai have shown that separate 
Wyner-Ziv coding followed by Gelfand-Pinsker coding is optimal for this problem. However, we show that there 
is a joint source-channel coding scheme for the case of Gaussian source, interference and side information. This 
result in Section IV is a fairly straightforward extension of the results in Section [IT] and Section III but is included 



for completeness and to make the exposition clear. In Section [V| we study the performance of these schemes when 
the SNR of the channel is different from the designed SNR and in Section [VI] the distortion exponents of these 



schemes are analyzed. In Section VII we consider the problem of transmitting a Gaussian source in the absence 



of an interference, but when the channel bandwidth is smaller than the source bandwidth and show how the HDA 



Costa coding scheme is useful. Finally, in Section VIII we consider the problem of broadcasting a Gaussian source 
to two users through AWGN channels and propose a joint source channel coding scheme based on HDA Costa 
coding. 

We use the following notation in this paper. Vectors are denoted by bold face letters such as x. Upper case letters 
are used to denote scalar random variables. When considering a sequence of i.i.d random variables, a single upper 
case letter is used to denote each component of the random vector. 

II. Transmission of a Gaussian source over a Gaussian channel with interference known only 

AT THE TRANSMITTER 



S W 



Encoder 



X 



Decoder 



Fig. 1. Block diagram of the joint source channel coding problem with interference known only at the transmitter. 

We first consider the problem of transmitting N samples of a real analog source v £ ~R N (this corresponds to 
K = N) , with components V which are independent Gaussian random variables with V ~ A/"(0, &%) in N uses 
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of an AWGN channel with noise variance a 2 in the presence of an interference s G M. N which is known to the 
transmitter but unknown to the receiver. Further, let us assume that S"s are a sequence of real i.i.d Gaussian random 
variables with zero mean and variance Q and let the input power to the channel E[X 2 ] be constrained to be P. 
The problem setup is shown schematically in Fig. [T] The received signal y is given by 



x + s + w (1) 
The optimal distortion of , av p ^ can be obtained even in the presence of the interference by using the following 

V c 



where s is the interference and w is the AWGN 
The optimal distortion of , ff "„ \ can be obtaii 

(obvious) separate source and channel coding scheme. 



A. Separation based scheme with Costa coding (Digital Costa Coding) 

We first quantize the source using an optimal quantizer to produce an index m G {1, 2, . . . , 2^}, where 
R = \ log (l + ^) — e. Then, the index is transmitted using Costa's writing on dirty paper coding scheme [9]. 
Since the quantizer output is digital information, we refer to this scheme as digital Costa coding. We briefly review 
this here to make it easier to describe our proposed techniques later on. 

Let U be an auxiliary random variable given by 

U = X + aS (2) 

where X ~ Af(0, P) is independent of S and a = pj^a ■ 

We first create an iV-lengfh i.i.d Gaussian code book U with 2 N ^ I( - U]Y ^ S ^ codewords, where each component 
of the codeword is Gaussian with zero mean and variance P + a 2 Q. Then evenly (randomly) distribute these over 
2^R bins. For each u, let i(u) be the index of the bin containing u. For a given m, we look for an u such that 
i(u) = m and (u, s) are jointly typical. Then, we transmit x = u — as. Note that since (u, s) are jointly typical, 
from (|2]), we can see that x_Ls and satisfies the power constraint. 

The received sequence y is given by 

y = x + s + w (3) 

At the decoder, we look for a u that is jointly typical with y and declare i(u) to be the decoded message. Since 
R = \ log (l + J^) — e, the distortion in v given by D(R), where D is the distortion rate function. For a Gaussian 
source and mean squared error distortion D(R) = a 2 2~ 2R and, hence, the overall distortion can be made to be 
arbitrarily close to jj^p ^ by a proper choice of e and 5. 

While the above scheme is straightforward, in the following three sections we show that there are a few other 
joint source channel coding schemes, which are also optimal. In fact, there are infinitely many schemes which 
are optimal. Although, these schemes are all optimal when the channel SNR is known at the transmitter, their 
performance is in general different when there is an SNR mismatch. The joint source channel coding schemes to 
be discussed in the next sections have advantages over the separation based scheme discussed in such a situation. 



B. Hybrid Digital Analog Costa Coding 

Let us now describe a joint source-channel coding scheme where the source v is not explicitly quantized. We 
refer to this scheme as hybrid digital analog (HDA) Costa coding for which the code construction, encoding and 
decoding procedures are as follows. 

We first define an auxiliary random variable U given by 

U = X + aS + kV (4) 

where X ~ M(0, P) and X, S and V are pairwise independent. 

1) Codebook generation: Generate a random i.i.d code book U with 2 NRl sequences, where each component of 
each codeword is Gaussian with zero mean and variance P + a 2 Q + n 2 a 2 . 

2) Encoding: Given an s and v, find a u such that (u, s, v) are jointly typical with respect to the distribution 
obtained from the model in Q and transmit x = u — as — kv. If such an u cannot be found, we declare an 
encoder failure. Let P ei be the probability of an encoder failure. 
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From standard arguments on typicality and its extensions to the infinite alphabet case [15], it follows that 
oo provided 



P ei -> as N 



Ri > I(U;S,V) 

= h{U) - h{U\S,V) 

= h(U) - h(X\S,V) 

= h{U)-h{X) 

1, P + a 2 Q + K 2 a 

= — log 

2 s P 



(5) 
(6) 
(V) 
(8) 

(9) 



where the results follow because X = U — aS — kV and X.LS, V. Notice that when a u that is jointly typical 
with s and v is found, x satisfies the power constraint. 
3) Decoding : The received signal is y = x + s + w. At the decoder, we look for an u that is jointly typical 
with y. If such a unique u can be found, we declare u as the decoder output or, else, we declare a decoder 
failure. Let P e2 be the probability of the event that the decoder output is not equal to the encoded u (this 
includes the probability of decoder failure as well as the probability of a decoder error). 
In order to analyze P e2 , consider the equivalent communication channel between U and Y. Notice that we 
have in effect transmitted a codeword u from a random i.i.d codebook for U with 2 NRl codewords through 
the equivalent channel whose output is y. Again, from the extension of joint typicality to the infinite alphabet 



case, P e2 — > as N — ► oo provided that 



I(U;Y) 
I(U;Y) 



> 



Ri 

h(U) 
h(U) 
h(U) 
h(U) 



h{U\Y) 
h(U-aY\Y) 

h(X + aS + kV — aX — aS 
H(kV + (1 - a)X - aW\Y) 



aW\Y) 



Now, let us choose 



a 



P + a 2 

p2 



(P + a 2 )a 2 a 2 

For the above choice of a, it can be seen that 

E[(kV + (1 - a)X - aW)Y] 



(10) 

(11) 
(12) 



and, hence, (10) reduces to 



I(U;Y) = h(U) - h(nV + (1 - a)X - aW) 

1 P + a 2 Q + n 2 a 2 v 
= - log 

2 6 P-e 



Hence, P e2 can be made arbitrarily small as long as 



1 P + a 2 Q + K 2 a 2 
RK- 2 loE ^ 



(13) 



(14) 



Combining this with the condition for encoder failure, P ei and P e2 can both be made arbitrarily small provided 

(15) 

z r — t 

7-» i _ 2 s~\ i . .2 _2 

I log 



1 P + a 2 Q + K 2 a 2 _ 1 P + a 2 Q + K 2 a 2 
2 l0g P < Rl < 2 l0g P^e 



Therefore, by choosing an ei, < ei < e and Ri 



and Pp. 



as N 



P-e! 



we can satisfy ( 15 ) and make P ei — > 



oo. 



4) Estimation: If there is no decoding failure, we form the final estimate of v as an MMSE estimate of v from 
[y u]. After some algebra this is given by , 
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v = _^-( u -ay) (16) 

The distortion is then given by, 

E[{V - V?) = -^V-^- < -^V + 5(6) (17) 

with (5(e) is vanishing for arbitrarily small e. If an encoder or decoder failure was declared, we set the estimate 
of v to be the zero vector. However, as shown above the probability of these events can be made arbitrarily 
small and, hence, they do not contribute to the overall distortion, which can be seen to be arbitrarily close 
to the optimal distortion achievable in the absence of the interference. 
We have presented a joint source channel coding scheme in the presence of an interference known only to the 
transmitter. The use of the term hybrid digital analog Costa coding needs some explanation. The scheme is not 
entirely analog in that the auxiliary random variable is from a discrete codebook. However, in contrast to digital 
Costa coding, the source is not explicitly quantized and is embedded into the transmitted signal x in an analog 
fashion. This is the reason for calling this as HDA Costa coding and this has some interesting consequences which 
are discussed in the following section. 

Another feature of the HDA Costa coding scheme is that it does not make use of binning, rather it needs a single 
quantizer codebook that is also a good channel code. In practice, this may have some impact on the design since 
there are ensembles of codes that are provably good quantizers and channel codes [13]. In the Gaussian case, good 
lattices that are both good for coding and for quantization are known. The binning approach however, requires a 
nesting condition. That is, the fine code much be a good channel code, but it much contain a subcode (or a coarse 
code) and its cosets that must be good quantizers. This may be a more difficult condition to obtain in practice. 

C. Superimposed digital and HDA Costa coding scheme 







V* 




Costa 




Source 






Encoder 


V 








Encoder 
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HDA Costa 










Encoder 




Fig. 2. Encoder model for superimposed coding 

Recently in [6], Bross, Lapidoth and Tinguely considered the problem of transmitting N samples of a Gaussian 
source in N uses of an AWGN channel, in the absence of the interferer. They showed that there are infinitely 
many superposition based schemes, which contain pure separation based scheme and uncoded transmission as 
special cases. In this section, we show that the same is true in the presence of an interference also and show the 
corresponding scheme, which is given in Fig. [2] 

The transmitted signal is a superposition of two signals x c and Xh c , which are the outputs of a digital Costa 
encoder and an HDA encoder, respectively. 

The source is first quantized at a rate of R < C using an optimal source code and let the quantization error 
be e = v — v*, where v* is the reconstruction. The quantization error e has a variance a\ = a%2~ 2R . The first 
stream in Fig. [2] is a digital Costa encoder that encodes the quantization index by treating s as interference and 
produces the signal x c , which has a power of Pc- The second stream is a HDA Costa encoder of rate R which 
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treats s and x c as interference and produces Xh c , which has a power of Phc = P ~ Pc- The transmitted signal is 
the superposition (sum) of x c and Xh c - 

In the digital Costa encoder in the first stream, the auxiliary random variable is given by u c = x c + a c s with 
x c _Ls. A power of Pc = (P + c 2 )(l — 2~ 2R ) is used in the first stream and a c is chosen as p + p^ c+a2 ■ Note 
that this corresponds to treating Xh c as noise in addition to the channel noise. 

In the second stream, the quantization error e is encoded using an HDA Costa coding scheme and a power of 
Phc = P ~ Pc = (P + cr 2 )2~ 2R — a 2 is used. Note that since R < C, the power Phc is always positive. The 
auxiliary random variable is chosen as Uh c = Xh c + a/i C ( x c + s) + ne, where x c + s acts as the net interference. 
Hence, Xh c is chosen to be independent of x c , s and e, and a,h c is chosen to be P ^"+ a 2 ■ k is chosen similar to 



<|12| which gives k 2 = {Ph?+(j h 2 ^ 12 ^ r - ^pb*. 

/It the decoder the quantization index from the first stream is first decoded and the reconstruction v* is obtained. 
Then, an estimate of the quantization error e is obtained from the second stream using the HDA costa decoder. The 



overall distortion is the distortion in estimating e. Using the analysis of the HDA Costa scheme in Section II-B 
this can be seen to be 

D = - , + ^ = ttV + ^ (18) 

1 H ^2 <T 2 

By choosing e to be arbitrarily small we can make 5(e) — ► and achievable a distortion of D = °" P , which is 
the optimal distortion. 

Note that for any source coding rate chosen in the first stream namely R, the resulting distortion is optimal. By 
varying R, we can get an infinite family of optimal joint source channel coding schemes. 

D. Generalized Hybrid Costa coding 

In the previous section, we described a superposition technique. In this section we show a scheme that does not 
explicitly do superposition. Moreover this also introduces an interesting scheme that is intermediate between HDA 
Costa having no bins to the digital Costa having bins corresponding to the capacity of the channel. 

Once again we quantize the source v to v* at a rate R, that is strictly lesser than the channel capacity, using an 
optimal vector quantizer. Let e = v — v* be the quantization error vector. Note that for an optimal quantizer, as 
the Rate-Distortion limit is approached, the quantization error e will be Gaussian. 

We next define an auxiliary random variable U given by 

U = X + aS + K 1 E (19) 

where X ~ N(0, P),E ~ Af(0, a 2 2~ 2R ), and X, S and E are independent of each other, a and K\ are constants, 
the choice of which is discussed below 

1) Codebook generation: Generate a random i.i.d code book U with 2 NI ( U '< Y ^ sequences, where each component 
of each codeword is Gaussian with zero mean and variance P + a 2 Q + n\a 2 2~ 2R . These codewords are 
uniformly distributed in 2^ bins and this is shared between the encoder and the decoder. 

2) Encoding: Let m be the quantization index corresponding to the quantized source v*. Let i(u) represent the 
index of a bin that contains u. For a given m find an u such that i(u) = m and (u, s,e) are jointly typical 



with respect to the distribution in model ( 19 1. We next transmit the vector x = u — as — K\e. Note that since 
(u,s,e) are jointly typical, from (19 1, we can see that x_Ls,e and satisfies the power constraint. 
3) Decoding : The received signal is y = x + s + w. At the decoder, we look for an u that is jointly typical 
with y. If such a unique u can be found, we declare u as the decoder output or, else, we declare a decoder 
failure. Next we make an estimate of e from u and y. 

We can see by similar Gelfand-Pinsker coding arguments that R < I(U ; 1") — I(U ;S,E). Note 
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I(U;Y) - I(U;S,E) 

= h(U\S,E) - h(U\Y) 

= h(X) — h(U — aY\Y) 

= h(X) - h{niE + (1 - a)X - aW\Y) 



(a) 



h(X) - h( Kl E + (1 - a)X - aW) 
bg (k?(t22- 2 « + (1 - a) 2 P + a 2 a 2 1 (20) 



5 R 



T /OA, u P A 2 P (P+a 2 )-a 2 2 2R e{{P+a 2 )-a 2 2 2R ) „, ,. f 

In (20 1 we choose a = P + a i and k| = p ^ ct2 ^ p^i The choice of a ensures 

(1 — a)X — aW is orthogonal to Y to get the equality in (a). k\ is chosen as above to satisfy the inequality 
in (b). This shows that we can decode the codeword u with a very high probability and we can decode the 
message m = i(u) and v*. 

4) Estimation: If there is no decoding failure, we form the final estimate of v as an MMSE estimate of v from 
[v* u y]. The estimate can be obtained as follows. Let us define a 2 e = a%2~ 2R . Let A be the covariance 
matrix of [V* U Y] T and let T be the correlation vector between V and [V* U Y] T . Then, A and T are 



given by 



al -al 




P + K 2 a 2 + a 2 Q P + aQ and T = ( a 2 - a 2 K X a 2 e ) T 
P + aQ P + Q + a 2 ) 

The coefficients of the linear MMSE estimate are given by A _1 T and the minimum mean-squared error is 
given by 

d = a 2 — r T A -1 r = 

where 5(e) — > as e — > 0. Thus, in the limit of e — > 0, D 

It must be noted that this scheme is an intermediate between digital Costa coding scheme with the maximum 
possible bins equal to the capacity of the channel and the analog Costa coding scheme with no bins. Thus 
we can get a family of schemes with varying bins for the Gaussian channel. 

The generalized hybrid Costa coding scheme appears to be closely related to the superimposed digital and 
HDA Costa coding schemes. The subtle difference however is in the generalized hybrid Costa coding scheme, 
the transmitted signal X is not a superposition of two streams as seen in the superposition case. 

III. Transmission of a Gaussian source through a channel with side information available 

ONLY AT THE RECEIVER 

In this section we consider the problem of transmitting a discrete-time analog source over a Gaussian noise 
channel when the receiver has some side information about the source. This problem is a dual of the problem 
considered in the previous section and is considered here for the sake of completeness. Consider the system model 
as follows. Let v G M. N be the discrete-time analog source where Vs are independent Gaussian random variables 
with V ~ J\f(0,a 2 ). Let v' G R be the side information that is known only at the receiver. The correlation 
between the source and the side information is modeled as 

V = V' + Z (21) 

where Z ~ Af(0, a 2 ) and V is i.i.d Gaussian. Here V' and Z are mutually independent random variables. The 
source v must be encoded into x and transmitted over an AWGN channel and the received signal is 

y = x + w (22) 
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where x satisfies a power constraint P and w is AWGN having a noise variance of a 2 . The following schemes 
can be shown to be optimal for this case. 



A. Separation Based Scheme with Wyner Ziv Coding (Digital Wyner Ziv Coding) 

One strategy is using a separation scheme with an optimal Wyner-Ziv code of rate R followed by a channel code. 
We also refer to this scheme as the digital Wyner-Ziv scheme. We briefly explain the digital Wyner-Ziv scheme 
and then establish our information theoretic model for the HDA Wyner-Ziv coding scheme. 



V 


Encoder 


X 







Y 


Decoder 









V 



V 



Fig. 3. Block diagram of the joint source channel coding problem with side information known only at the receiver. 

Suppose the side information is available both at the encoder as well as the receiver, the best possible distortion 
is D = °* P . The same distortion can be achieved using the following scheme and is a direct consequence of 
Wyner and Ziv's result [16]. This can be achieved as follows, 

Let U be an auxiliary random variable given by 

U = JaV + B (23) 

where a = 1 - ^ = and B ~ A/"(0, D). We create an TV-length i.i.d Gaussian code book U with 2 NI{U ' V ^ 

codewords, where each component of the codeword is Gaussian with zero mean and variance aa 2 + D and evenly 
distribute them over 2^ bins. Let i(u) be the index of the bin containing u. For each v, find an u such that 
(u, v) are jointly typical. The index i(u) is the Wyner-Ziv source coded index. The index i(u) is encoded using 
an optimal channel code of rate arbitrarily close to | log(l + ^) and transmitted over the channel. At the receiver 
decoding of the index i(u) is possible with high probability as an optimal code book for the channel is used. Next 
for the decoded i(u) we look for an u in the bin whose index is i(u) such that (u, v') are jointly typical. From 
v' and the decoded u we make an estimate of the source v as follows. 

v = v' + y/a(u - y/av 1 ) (24) 

This yields the optimal distortion D. 



B. Hybrid Digital Analog Wyner Ziv Coding 

In this section, we discuss a different joint source channel coding scheme that does not involve quantizing the 
source explicitly. This scheme is quite similar to the modulo lattice modulation scheme in [2]; the difference being 
that a nested lattice is not used. The auxiliary random variable U is generated as follows. 

U = X + kV (25) 

where k is defined as k 2 = rp^sj^i — ^ and X ~ A/"(0, P). 

1) Codebook generation: Generate a random i.i.d code book U with 2 NRl sequences, where each component 
of each codeword is Gaussian with zero mean and variance P + K 2 a 2 . This codebook is shared between the 
encoder and the decoder. 

2) Encoding: For a given v find an u such that (u,v) are jointly typical and transmit x = u — kv. This is 
possible with arbitrarily high probability if R\ > I(U ; V) 
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3) Decoding: The received signal is y = x + w. Find an u such that (v',y,u) are jointly typical. A unique 
such u can be found with arbitrarily high probability if R\ < I(U ; V, Y). We next show below that we can 
choose an Ri to satisfy I(U; V) < Ri < I(U;V',Y). This requires I(U; V) < I(U;V',Y) which can be 
shown as follows 



I(U;V,Y) = h(U)-h(U\V',Y) 

= h(U) - h(U - kV' - aY\V \Y) 
= h(U) - h(nZ + (1 - a)X - aW\V , Y) 

( = } h(U) - h{ K Z + (1 - a)X - aW) 

1 , ( P + k 2 oI 

2 ° g \K 2 a 2 z + {1 - a) 2 P + a 2 a 2 
(b) 1 fP + K 2 a^\ 

= h{U) - h(U\V) + 5{e) 

= I(U;V) + 5(e) (26) 



In (26), (a) follows because (kZ + (1 — a)X — aW) is independent of Y and V . (b) follows because we 
can always find a 5(e) > for the choice of k 2 = rpq~2j^ — Hence from knowing v', u and y we 
can make an estimate of v. Since all random variables are Gaussian, the optimal estimate is a linear MMSE 
estimate which can be computed as follows. 

Let A be the co variance matrix of [V' U Y] T and let T be the correlation between V and [V' U Y] T . A 
and r are given by 

al - a 2 z k(ct 2 - a 2 ) 
A=| K(a 2 -a 2 z ) P + K 2 a 2 P ) and T = ( a 2 - a 2 ku 2 v f 
P P + a 2 

The coefficients of the linear MMSE estimate are given by A~ 1 T and this yields the optimal MMSE estimate 
which is given below as, 

KG 2 

v = v' + -^(u-CT'-ay) (27) 

The distortion D is given by 

D = £[(v-v) 2 ] 

= El(z-^(Kz + x-ay)) 2 ] 

= M((i-^-^((i-«)x-H) 2 l 

(a) a 2 



i + S +5(£) 



(28) 



Here, (a) follows by using the appropriate values of k and a. We once again obtain the optimal distortion D 
by making e arbitrarily small and 5(e) — > 0. 
It is instructive to compare the performance of this scheme with that of the following naive scheme that would be 
optimal in the absence of side-information at the receiver. In the naive scheme, the v is transmitted directly (analog 
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transmission). At the receiver, an MMSE estimate of v is formed from the received signal y and the available side 

al/ {I + (P/a 2 )a 2 z ). 



information v'. The distortion for this naive scheme can be seen to be D na i ve 

= o = l, whereas for the Wyner-Ziv scheme, ^\ a 2 =0 

D 



Notice that =?p 



i 



< 1. At at 



0, both 



, , and D are zero. i.e. the optimal scheme and the naive scheme approach zero distortion with different slopes. 



C. Superimposed digital and HDA Wyner-Ziv scheme 

The above results could also be extended to a form of superimposed digital and analog coding. This is similar to 



the superimposed digital and HDA Costa coding case discussed in section II-C We once again have two streams as 
shown in Fig |4] The first stream uses a rate R Wyner Ziv code to quantize the source assuming the side information 
v' is known at the receiver and the discrete index is encoded using an optimal channel code to produce the codeword 
xi. The power allocated to this stream is Pvyz = (P + o~ 2 )(^ — 2~ 2R ). The second stream uses the HDA Wyner-Ziv 
scheme and produces the output X2- The auxiliary random variable of the HDA scheme is given by 



U = k 1 V + X 2 



(29) 



with X 2 ~N{Q,Phwz), where P HWZ = (P + a 2 )2 
4, where a 2 , = a 2 2~ 2R . 



-2R 



a 2 and X 2 and V are independent. We also choose 



1 ~~ (Phwz+ct*)^ 

The two streams (xi and X2) are superimposed and transmitted through the channel. The received signal is given 
by y = xi + X2 + w. At the receiver xi is decoded assuming X2 + w as independent noise and this gives the 
Wyner-Ziv encoded bits (index). This along with the side information v' can be used to make an estimate of the 
source v and we call the estimate as v. The random variables corresponding to v and v are related as 



V = V + Z 



(30) 



with Z having a variance a 2 2~ 2R . When the digital part is first decoded and canceled from the received signal, we 
get an equivalent channel for the HDA Wyner-Ziv scheme with power constraint Phwz and channel noise a 2 . We 
next make a final estimate of v using a HDA Wyner-Ziv decoder from the new side information v, the observed 
equivalent channel (y — xi) and the decoded u. Notice that since the choice of k\ = rp^pffsj^ — <r* wriere 
2 _ , T 22-2i? j s designed f or tne side information v, this ensures decoding of u with arbitrarily high probability. 



cr. 



a 



The achievable distortion is then given as follows. 



V . 


Digital 
Wyner-Ziv 
Source encoder 


R bits 


Channel Coder 


Xl 











x 



HDA 
Wyner-Ziv 
Encoder 



x 2 



Fig. 4. Block diagram of the encoder of the superimposed digital and HDA Wyner-Ziv scheme. 



2 



Phwz + o l 



(b) 2 p HWZ + o- 2 a 2 

P + o 2 Phwz + ° 2 



(31) 
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Here in ( 3 1 ) (a) follows since we assume that the first stream is decoded with high probability and apply the results 
of HDA Wyner-Ziv decoding with the side information v. Also (b) follows since Phwz = (P + a 2 )2~ 2R — a 2 . 
The optimal distortion °* P can be obtained by making e arbitrarily small and 5(e) — ► 0. Notice that for any rate 
R, < R < C, where C is the capacity of the AWGN channel, there is a corresponding power allocation for 
Phwz = (P + <J 2 )2~ 2R — a 2 for which the overall scheme is optimal. Thus, there are infinitely many schemes 
which are optimal with the digital Wyner-Ziv corresponding to Phwz = and the HDA Wyner-Ziv corresponding 
to Phwz = P and R = 0. 

Further, we would like to mention that there is another way to get a family of optimal schemes using the HDA 
Wyner-Ziv scheme. Here, the source v is encoded using a HDA Wyner-Ziv encoder to the sequence x. The auxiliary 
random variable U is given by 

U = kV + X (32) 

where n = rpq^iyi — ^s- The sequence x can be treated as an i.i.d Gaussian source and, hence, the family of 
schemes proposed by Bross, Lapidoth and Tinguely [6] can be applied on x. The scheme proposed in [6] quantizes 
the analog source, which in this case is x to a quantization index and is sent over the Gaussian channel along with 
the uncoded analog source (here x) with the appropriate power scaling. At the receiver we can obtain an optimal 
estimate of x by first decoding the quantized index and then making an estimate on the analog source. Notice 
that the HDA Wyner-Ziv receiver only requires an optimal MMSE estimate of x, which can be obtained using the 
family of schemes in [6]. Hence the resulting distortion in v is still optimal. To establish this claim we need to 
show that u can be decoded with arbitrarily high probability and an optimal estimate of v must be made using u 
and the MMSE estimate x. 

We next show below that I(U ; V) < I(U ; V ,X). Hence, we can choose a codebook for u with 2 nRl codewords 
such that I(U; V) < R\ < I(U; V',X). Since I(U; V) < Ri, we can find a u that is jointly typical with v with 
probability close to 1 and since i?i < I(U; V',X), u can decoded with high probability from (V',x). 

I(U;V',X) = h(U) - h(U\V , X) 

= h(U) - h(U - kV' - X\V' , X) 
= h(U) -h(nZ + X-X\X,V') 



(a) 



h(U) - h(nZ + X - X) 



(b) 1 ( P + K 2 a 2 
log 



2 \K 2 a 2 + aa 2 

2 lo H^^J + (e) 

h(U) - h(U\V)+5(e) 

I(U;V) + 6(e) (33) 



In (33 1, (a) follows because (X — X) is orthogonal to X and hence (kZ + X — X) is independent of X and 
V, (b) follows because X — X is Gaussian with variance aa 2 and is orthogonal to Z. The estimate of v is then 
given by 

v = v' + ^(u-ct'-x) (34) 



The resulting distortion can be obtained by following the steps similar to those in (28 1 which can be found to be 
optimal. 

IV. Transmission of a Gaussian Source with Interference at the Transmitter and Side 

Information at the Receiver 

In this section, we consider the problem of transmitting a Gaussian source v through an AWGN channel with 
channel noise variance a 2 in the presence of an interference s known only at the transmitter and in the presence 
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of side information v' known only at the receiver. The side information v' is assumed to be related to the source 
v according to 

v = v' + z 

where Z ~ A/"(0, a 2 ) and is independent of V' . 

A similar model has been considered by Merhav and Shamai [11] for a more general setup where the source 
and side information are not assumed to be Gaussian. They show that a separation based approach of Wyner-Ziv 
coding followed by Gelfand-Pinsker coding is optimal. Here, we propose a joint-source channel coding scheme 
when the source and channel noise are Gaussian. The proposed scheme is easily obtained by combining the results 
from the previous two sections. It must be noted that a similar joint source channel coding scheme using nested 
lattices and dither has been shown in [7]. However, our scheme is based only on random code books. 

To establish our scheme we can combine the results from the previous two sections as follows. Choose the 
auxiliary random variable U such that 

U = X + aS + kV (35) 

with k 2 = {pj ^2 )rj2 - -§s and a = Further, let X ~ Af(0,P), S ~ jV(0, Q) and V ~ Af(0,o%) and let 

X, S and V be pairwise independent. A codebook U is obtained by generating 2 1 code sequences for u and 
this is shared between the encoder and decoder. At the encoder, the source v is encoded by choosing an x that is 
jointly typical with u,v and s. Such a u exists with high probability if we have chosen R\ > I(U ; S, V). Now x 
is transmitted over the channel. The received signal vector y is given as 

y = x + s + w 

At the decoder, u is decoded by looking for a u that is jointly typical with y and the side information v'. 
Using standard arguments on joint-typicality, it can be seen that a unique such u exists with high probability if 
Ri < I(U;Y,V). We now show that I(U;S,V) < I(U;Y,V). This implies that there exists an R x , such that 
I(U ; S, V) < R\ < I(U; Y, V') which satisfies the requirements at the encoder and the decoder. 



I(U;Y,V) = h(U) - h(U\Y,V) 

= h(U) - h{U - aY - kV'\Y,V') 

= h{U) - h{nZ + (1 - a)X - aW\Y,V) 

h(U) - h{nZ + (1 - a)X - aW) 

1 P + aPQ + K 2 ** 

2 lo §( p ) + <H £ ) 

I(U; S, V) + 5{e) 



(a) 



(36) 



where (a) follows since kZ+(1 — a)X — aW is orthogonal to Y and V. Then an optimal linear MMSE estimate 
of v is formed from the side information v', the received vector y and the vector u. By using the argument as in 
section. III-B the MMSE estimate is given by 

J2 



V + 



kg; 
P 



(u 



KV 



ay) 



(37) 



The resulting distortion can be obtained by following steps similar to (28 1 and can be seen to be D 



which is the optimal distortion. 



V. Analysis of the schemes for SNR mismatch 

In this section, we consider the performance of the above JSCC schemes for the case of SNR mismatch where 
we design the scheme to be optimal for a channel noise variance of a 2 , but the actual noise variance is a 2 . 

Separation based digital schemes suffer from a pronounced threshold effect. When the channel SNR is worse 
than the designed SNR, the index cannot be decoded and when the channel SNR is better than the designed SNR, 
the distortion is limited by the quantization and does not improve. However, the hybrid digital analog schemes 
considered offer better performance in this situation. 
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Let us consider the joint source channel coding setup with side information at both the transmitter and receiver 
and a 2 < a 2 . We can decode u at the receiver when the SNR is better than the designed SNR and make an estimate 
of the source from the various observations at the receiver as shown below. 



30 r 



25 ■ 



5 - 



HDA Costa - Source 




Digital Costa - Interference 

V" 



Gen. HDA Costa - Soi 



Gen. HDA Costa - Interference 
HDA Costa - Interference 





10 15 20 25 30 35 40 45 50 

10 Iog10(1/o 2 ) 



Fig. 5. Performance of the different Costa coding schemes for the joint source channel coding problem. 



U = X + aS + K W V 

v = v' + z 

Y = X + S + W a 



(38) 
(39) 
(40) 



where k w = J » oc = P + a i , S ~ Af(0, Q) and Z ~ M(0, a z ). From now on, we drop the e's in k w to 

improve clarity. Note that a depends only on the assumed noise variance a 2 and not on a 2 . 

From t he obs ervations [V', U, Y], an optimal linear MMSE estimate of V is obtained. Similar to the definition 
let A be the covariance of [V' , U, Y] T and T be the correlation between V and [V, U,Y] T . 



in section 
Hence 



III-B 



k(<j. 



v 



K(a 2 - a 2 ) 



P + a l Q + k 2 w o 2 





P + aQ 



P + aQ P + Q + a 2 a 

Then the distortion (in the presence of mismatch) is given by 



and r = ( o 2 -o\ 



koI 



D„ 



r T A- x r 



(41) 



This on further simplification yields 



D a = [{Q<J A + (P(P + Q) + 2Pa 2 + a 4 )a 2 a )a 2 } x 
[P 2 {P + Q) + P{P + Q)a 2 + Qa 4 + (P(2P + Q) + 3Pa 2 + a 4 )a 2 a ] _1 
Let us now look at a few special cases 



(42) 
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A. Hybrid Digital Analog Costa Coding 

In this setup there is side information only at the transmitter. The distortion achievable for the user under SNR 



mismatch with the actual SNR greater than the designed SNR is obtained by setting a v = a z (42 1 and is given 
below. 

D va = [{Qa 4 + (P(P + Q)+ 2Pa 2 + a 4 )a 2 a )a 2 ] x 

[P 2 {P + Q) + P{P + Q)a 2 + Qa 4 + (P(2P + Q) + 3Pa 2 + a 4 )a 2 a ] -1 ' 4 ' ' 

The distortion in the source v is shown in Fig [5] for a designed SNR of 10 dB as the actual channel SNR 
(10 log 1/cr 2 ) varies when the source and interference both have unit variance. It can be seen that the distortion in 
the source is smaller with the HDA Costa scheme than with the digital Costa scheme. 

In some case, the distortion in estimating the interference at the receiver may also be of interest and can be 
obtained by estimating S from ( [38] ) and ( |40"| ). The distortion is given below, 

D sa = [Q(P + a 2 )(P 2 + (2P + a 2 )a 2 a )} x 

(44) 

[P 2 (P + Q) + P(P + Q)a 2 + Qa 4 (P(2P + Q) + 3Pa 2 + a 4 )a 2 a ] 

It can be seen from Fig. [5] that the distortion in estimating the interference is better for the digital scheme than 
for the HDA Costa scheme. 

In [14], Sutivong et al. have studied a somewhat related problem. They consider the transmission of a digital 
source in the presence of an interference known at the transmitter with a fixed channel SNR. They study the optimal 
tradeoff between the achievable rate and the error in estimating the interference at the designed SNR. The main 
result is that we can get a better estimate of the interference if we transmit the digital source at a rate lesser than the 
channel capacity. There are important differences our work and that in [14]. First of all, we consider transmission of 
an analog source instead of a digital source. Secondly, we consider mismatch in the channel, i.e., our schemes are 
designed to be optimal at the designed SNR and as we move away from the designed SNR, we study the tradeoff 
between the error in estimating the interference and the distortion in the reconstruction of the analog source. This 
tradeoff is discussed below. 

B. Generalized HDA Costa Coding under channel mismatch 

Next we analyze the performance of the generalized HDA Costa coding under channel mismatch. This case leads 
to some interesting analysis. By changing the source coding rate of the digital part R, we can tradeoff the distortion 
between the source and the interference in the presence of mismatch. 

The different random variables and their relations are given below. 

U = X + aS + KiE (45) 
Y = X + S + W a (46) 
V = V* + E (47) 



In the above equation k\ = J p ^_ a2 ^ (Again, we have dropped the e in the expression for k\ .) From 

the above equations an estimate of S as well as V is obtained by taking a linear MMSE estimate as all the random 
variables are Gaussian. The resulting expressions of estimation error D sa (R) and D va (R) are given by 

D va (R) = [(a 2 a (a 2 + P) 2 + (a 4 + a 2 a P)Q)a 2 ] x 

[(^ 2 + P)\°l + P + Q) ~ 2 2i V - o\)P{a 2 + P + Q)Y 1 

D sa {R) = [{a 2 + P){2 2R {o 2 - a 2 a )P - {a 2 + P){a 2 a + P))Q] 

(49) 

x [2 2R {a 2 -a 2 a )P{a 2 + P + Q)-{a 2 + P) 2 {a 2 a + P + Q)Y l 
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The performance of the generalized HDA Costa scheme and HDA Costa scheme in relation to digital scheme 
is shown in fig. [5] For example in separation using digital Costa there is no improvement in our estimate of the 
analog source, but we get a better estimate of the interference as shown in figj5] On the contrary for the HDA 
Costa scheme there is only a small improvement in the estimate of the interference but a good improvement in the 
estimate of the analog source. The generalized HDA also shows a difference in the estimate for the source and the 
interference for different rates R and performs as a digital Costa for the choice of R = C and as HDA Costa for 
the choice of R = 0. In effect we can tradeoff the estimation error in interference with the source by choosing 
different values of R when there is a channel mismatch. 



C. Hybrid Digital Analog Wyner Ziv 



In this case the distortion could be obtained by setting Q = in (42 1. The actual distortion is given by 




14 16 
10 log 10(1 /a 2 ) 



Fig. 6. Performance of the different Wyner-Ziv schemes for the joint source channel coding problem. 



D - ( P + °»* (50) 

This is clearly better than which is what is achievable with a separation based approach. However, we 

don't know if this is the optimal distortion that is achievable in the presence of channel mismatch. A simple lower 
bound on the achievable distortion in the presence of mismatch is to assume that the transmitter knows the channel 
SNR. Based on this we can analyze the gap in dB between the distorion of HDA Wyner Ziv scheme and the lower 
bound as follows. 

The lower bound on D is given by 

Dlower = Z ; T^Ti o (51) 

Now the gap between the analog Wyner-Ziv and the bound at high SNR can be easily calculated as lim CTa _ i o Dl ^" !T . 
The gap in db, G is hence given by 



G = mog {pT^) (52) 

For example, if our designed SNR is say 10 db, for high SNRs, we loose at most G = 0.41 db which is fairly 
close to the outer bound as shown in Fig. [6] 
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VI. Distortion exponent for HDA Costa and Wyner-Ziv schemes 

In this section, we consider the performance of the HDA joint source-channel coding schemes for transmitting 
a Gaussian source through a Gaussian channel when the actual channel noise variance a 2 is not known, but it is 
known that the variance is always smaller than a 2 . Since we are interested in the performance of a single encoding 
scheme over a wide range of noise variances, a useful measure of performance is the rate of decay of the distortion 
as a function of the actual noise variance in the limit a 2 — > 0. More precisely, we define a distortion exponent as 

C = lim l0g(Z ^ )} 
,x2-o log(a2) 

where D(a 2 ) is the distortion when the noise variance is a 2 . Notice that this exponent is quite different from the 
distortion signal-to-noise ratio (SNR) exponent considered in [17], [18] for the case of slow fading channels. In [17] 
and [18], the rate of decay of distortion with average SNR is studied by allowing for a family of coding schemes, 
one for each average SNR. In contrast, we fix the encoding scheme here and consider the rate of decay with the 
actual channel SNR (i.e., there is no fading). 

An upper bound on the achievable £ can be obtained by assuming that a genie informs the transmitter of a a and 
the transmitter chooses an optimal encoding scheme for a noise variance of a 2 . Let us assume a general case where 
there is an interference s (S ~ M(0,Q)) which is known at the transmitter and some side information v' which 
is related to v according to V = V + Z, where Z ~ M(0,a 2 ) is known at the receiver. Then, the distortion for 
the genie-aided scheme is ° Z P since an optimal Wyner-Ziv encoder followed by an optimal Costa encoder can be 

chosen. In this case, the distortion exponent is 1. Notice that in the absence of any side information the distortion 
for the genie-aided scheme is °" P which also results in an exponent of 1. In the absence of any interference also, 

the achievable distortion is a * P and the exponent is 1. Thus, for any single encoding scheme, C < 1 both in the 

^1 

presence and absence of interference and/or side information. 



We will now consider the performance of the HDA schemes considered in Section IV If a joint source channel 
coding scheme is designed to be optimal when the noise variance is a 2 , then the distortion when the noise variance 
is a 2 is given by (from (42) in Section [v} 



D{a 2 a ) = [(Qa* + (P(P + Q) + 2Pa 2 + a')a 2 a )a 2 } x 

[P 2 {P + Q) + P{P + Q)a 2 + Qa 4 + (P(2P + Q) + 3Pa 2 + a^a 2 ] ~ l . ' ' 

We now consider two cases. 

A. Absence of Interference 

When there is no interference at the transmitter, Q = and, hence, from (53), we can see that the optimal 
distortion can be obtained for a noise variance of a 2 and the optimal distortion exponent of ( = 1 can be obtained. 
Thus, this scheme performs as well as the genie-aided receiver in the distortion exponent sense. 

B. Presence of Interference 



In the presence of an interference, Q ^ 0, and from (53), Q can be seen to be zero. That is, some amount of 
residual interference is always present and, hence, in the high SNR limit (a 2 — ► 0), the performance is dominated 



by this residual interference. However, if optimal performance is not desired when the noise variance is a 2 , then 



the optimal exponent of £ = 1 can be obtained using a minor modification to the scheme discussed in Section IV 
In the modified scheme, the auxiliary random variable U is generated as follows 

U = X + S + K e V (54) 
Note that a is chosen to be 1, which is clearly not optimal for a noise variance of a 2 . The side information V' is 

V = V' + Z (55) 

and the received signal is 

Y = X + S + W (56) 



17 



Using arguments similar to those in Section IV n e is chosen so as to satisfy I(U;Y,V) > I(U;S,V). The 
required condition on K e can be obtained as follows 

I(U;Y,V) > I(U;S,V) 
h{U) - h{U\Y,V') > h{U)-h{U\S,V) 

=>h(U\S,V) > h(U\Y,V) (57) 

Note that h(U\S, V) = h(X) and h(U\Y, V) = h(U -rjY- K e V'\Y, V), where 77 = §pj. For this choice of 
77, (U - rjY - K e V) _L Y, V and, hence, h(U\Y, V) = h(U - rjY - n e V). Hence, we get the relation, 

h(X) > h(U -rjY- KeV') 
^P > E[{U - rjY - K e V') 2 ] 



! P 2 + PQ- Qa 2 

(P+Q + (7>2 



> K P 



Hence, K e can be chosen to be arbitrarily close to y {p+q+ ct ^2 ■ Now x is transmitted and y is received. The 
optimal distortion is obtained as an MMSE estimate of v from [y,u, v']. The final distortion is given by 

H(« 2 \ = (P + QVM 

{ J (P + Q)o* + i$(P+Q + o*)o* P6j 

It can be seen that as a 2 — > 0, D(a 2 ) oc a 2 and, hence, ( = 1. 

VII. Applications to Transmitting a Gaussian source with Bandwidth Compression 

We now consider the problem of transmitting K samples of the i.i.d Gaussian source to a single user in N = XK 
(A < 1) uses of an AWGN channel with noise variance a 2 where the transmit power is constrained to 1. There is 
no interference in the channel, but since A < 1, we will see that the techniques described in the previous sections 
are useful for this problem. 

There at least three ways to achieve the optimal distortion in this case. One is to use a conventional separation 
based approach. The second one is to use superposition coding and the third one is to use Costa coding. Although, 
they are all optimal for the single user case, they perform differently when there is a mismatch in the channel SNR 
and, hence, the last two approaches are briefly described here. 

a) Superposition Coding: Here we split the source in two parts and take N samples of the source v, namely 
and scale it by ^fa creating the systematic signal xi = ^fav^ ■ We take the other K — N source samples 
v^ + i and use a conventional source encoder followed by a capacity achieving channel code resulting in the N 
dimensional vector x c = C(Q(v^ +l )), where C denotes a channel encoding operation and Q denotes a source 
encoding operation. Then x c is normalized so that the average power is \f\ — a. The overall transmitted signal is 
x = x s + x c and the received signal is y = x + w. At the receiver, the digital part is first decoded assuming the 
systematic (analog) part is noise and then x c is subtracted from y. Then an MMSE estimate of Vi is formed. For 
the optimal choice of a, the optimal overall distortion can be obtained given by 



* _ 2 

a sup ~ a 



and D* = =- (59) 

sup (i + _^ )A 

which is the optimal distortion. 

b) Digital Costa Coding: We split the source exactly as in the previous case and one stream is formed as 
x s = yfavi . However, here the digital part assumes that x s is interference and uses Costa coding to produce x c 
with power 1 — a as shown in Fig. [7] In Costa coding, we define an auxiliary random variable u = x c + aix s 
where a\ = 1 _ 1 aH a CT 2 is the optimum scaling coefficient. At the receiver, the digital part is decoded which means 
that u can be obtained. In spite of knowing u exactly, the optimal estimate of vf is obtained by simply treating 
x c as noise since for the optimal choice of a\, x c = u — aix s and are uncorrected. Therefore, an MMSE 
estimate of is formed assuming x c were noise. Hence, the overall distortion becomes 
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Fig. 7. Encoder model using Costa coding for single user 



Again, minimizing D w.r.t. a gives 



a casta = (1 + CT 



1 : r- and D* tn = r- (61) 

(1 + ^J + 

which is the best possible distortion. 

c) Hybrid Digital Analog Costa Coding: For the case of A = 0.5, the digital Costa coding part can be replaced 
by a hybrid digital analog (HDA) Costa coding. We refer to such a scheme as HDA Costa coding. The same power 
allocation however, remains the same and hence, we can simply use a* Costa without the need to differentiate the 
digital and HDA Costa coding. It is quite straightforward to show that a* Costa > a* sup for A < 1. Hence, the Costa 
coding approach allocates higher power to the systematic part than the superposition approach, since the systematic 
part is treated as interference. 

A. Performance in the presence of SNR mismatch 

Now, we consider the same set up as above, but when the actual channel noise variance is a 2 , whereas the 
designed noise variance is a 2 . 
Case 1: a 2 > a 2 

The distortion for the superposition code can be computed to be the sum of the distortions in the systematic part 
and the digital part. When a 2 > a 2 , the digital part cannot be decoded and, hence, we assume that the distortion 
in the digital part is the variance of the source, 1. 

D sup = ^— + (1 - A) • 1 (62) 

Both the digital and HDA Costa coding schemes perform identically when a 2 > a 2 and the distortion for the 
Costa code can be computed to be 

DdigCosta = D H DACosta = aj^ a h (1 — A) ■ 1 (63) 

1 + l-a'cZa+vl 

Case 2: a 2 < a 2 In this case, the digital part can be decoded exactly and, hence, the distortion for superposition 
coding is 

D «* = A + (1 - A) / J xvd-A) (64) 

For digital Costa coding, the decoder first decodes the digital part when the auxiliary random variable u is perfectly 
known. In the case when a 2 / a 2 , the receiver must form the MMSE estimate of from the channel observation 
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y and u. Therefore, the overall distortion is 

DdigCosta = \(l- Wa* Costa a/ 

J2 



a Costa\ X 



1 + < 1 - a* Costa + aa* Costa 

1 ~~ a Costa + aa Costa * ~~ a CWa + a a Costa 
V a Costa ^ - 1 



(65) 



a^/a 



Costa 



+ (1-A) 



1 + 



1— at. 



A/(l-A) 



For the HDA Costa coding, we can decode u and form MMSE estimates of and v^ +1 separately and, hence, 
the overall distortion is given by 



Dhdac. 



osta 



A (l " W a Costa a V a CosJ X 



1 + ^ 



1 ~ a Costa + aa C OSt « 
\/ a Costa 



1 a Costa + aa Costa 
a Costa + a2a *Costa + ^ 



-1 



ay/a 



+ (1 - A) (1 - [0 k] x 



Costa 



(66) 



1 - a Costa + aa Costa 1 





1 - a *Costa + aa C 



Costa 



l Costa + Costa 



The performance of the superposition scheme, digital Costa and HDA Costa scheme are shown for an example 
with A = 0.5 in Fig. 8 The designed SNR is defined as 101og 10 \ whereas the actual SNR is defined as 101og 10 \. 
In the example, the aesigned SNR is fixed at lOdB and the actual SNR is varied from dB to 20 dB. It can be 
seen that the Costa coding approach is better than superposition coding when a\ > a 2 and worse for the other 
case. The HDA Costa coding scheme performs the best over the entire range of SNRs. 
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Fig. 8. Performance of different schemes for the source splitting approach for the bandwidth compression problem with SNR mismatch. 



VIII. Applications to Broadcasting with Bandwidth Compression 

We now consider the problem of transmitting K = IN samples of a unit variance Gaussian source v in N uses 
of the channel to two users through AWGN channels with noise variances a\ (weak user) and a\ (strong user) with 
a\ > o"2. The channel has the power constraint P = 1. We are interested in joint source channel coding schemes 
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that provide a good region of pairs of distortion that are simultaneously achievable at the two users. This problem 
was considered in [1,5,8]. The best known region to date is given by the schemes therein. 

Notice that when we design a source channel coding scheme to be optimal for the weak user, the strong user 



operates under the situation of SNR mismatch explained in Section VII- A with a\ = a 2 < a 2 = a\. Similarly, 
when the system is designed to be optimal for the strong user, for the weak user a\ = a 2 > a 2 = a 2 . Motivated by 
the fact that for A = 0.5, the HDA Costa coding scheme performs the best, we propose a scheme which is shown 
in Fig. 

There are three layers in the proposed coding scheme. The first layer is the systematic part where N out of the 
K samples of the source are scaled by yfa. Let us call this as x s = y/avi ' . The other K — N samples of the 
Gaussian source are hybrid digital analog Costa coding, treating x s as the interference and transmits the signal xi 
with power b in the second layer. So xi = ui — a\x. s — k c v^ +1 , where a\ and k c are the optimal scaling coefficient 
to be used in the hybrid digital analog Costa coding process and ui is the auxiliary variable. This layer is meant 
to be decoded by the weak user and, hence, the scaling factor ol\ is set to be b/(b + c + a 2 ). That is, this layer 
sees the third layer also as independent noise. 
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Fig. 9. Encoder model using Costa coding 



The third layer is first Wyner Ziv coded at a rate R2 assuming the estimate of v^ +1 at the receiver as side 
information. The Wyner-Ziv index is then encoded using digital Costa coding assuming x s and xi are interference 
and uses power c = 1 — a — b. Therefore, X2 = 112 — 02 (x s + xi). This layer is meant for the strong user and, 
hence, the scaling factor a 2 = c/(c + a 2 ). We then transmit x = x s + xi + X2. 

At the receiver, from the second layer an estimate of v^ +1 is obtained. This estimate acts as side information that 
can be used in refining the estimate of vj§ +1 for the strong user using the decoded Wyner-Ziv bits. The Wyner-Ziv 



bits are decoded from the third layer by Costa decoding procedure. 

The users estimate the systematic part v± and non-systematic part v^ +1 by MMSE estimation from the received 
y, the decoded ui and 112. So the overall distortion seen at the weak user is 
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The distortion for the strong user is given by 

D 2 = - ( 1 - [y/a a\y/a\ 
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The corner points of the distortion region corresponding to being optimal for the strong and weak user respectively, 
can be obtained by setting c = and 6 = 0, respectively. 

The distortion region for this scheme for the case of a\ = dB and o\ = 5 dB is shown in Fig. 10 The 
distortion region for three other schemes are also shown. They are the scheme proposed by Mittal and Phamdo in 
[1], a different broadcasting scheme which uses digital Costa coding in both the layers proposed in [8] (details can 
be found there) and the broadcast scheme with one layer of superposition coding and one layer of digital Costa 
coding considered in [5,8]. This scheme currently appears to be the best known scheme. Notice that in the third 
layer, instead of using a separate Wynzer-Ziv encoder followed by a Costa code, we could have used the HDA 
scheme discussed in Section [IV] with identical results. 

The proposed broadcast scheme in Fig. [9] significantly outperforms the scheme in Mittal and Phamdo and the 
digital Costa based broadcast scheme for this example. The corner points of this scheme also coincide with those 
of the best known schemes reported in [5, 19]. 
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Fig. 10. Distortion regions of the different schemes for broadcasting with bandwidth compression. 



IX. Conclusion and future work 

We discussed hybrid digital analog version of Costa coding and Wyner-Ziv coding for transmitting an analog 
Gaussian source through an AWGN channel in the presence of an interferer known only to the transmitter and side 
information available only to the receiver respectively. These schemes are closely related to the schemes by Reznic 
and Zamir [2] and [7], but make the auxiliary random variable model more explicit. We also showed that there are 
infinitely many schemes that are optimal for this problem, extending the work of Bross, Lapidoth and Tinguely [6] 
to the side information case. The HDA coding schemes have advantages over strictly digital schemes when there 
is a mismatch in the channel SNR. This makes them also useful for broadcasting a Gaussian source to two users 
with different SNRs. 
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